Dynamic Properties of Neural Learning in the Information-theoretic Plane

Learning in reference to the real neural complex depicts progressive modifications occurring at the synaptic levels of th e interconnected neurons. The presence of int raneural dist urb ances (inherent ly present ) or any ext raneural noise in the input data or in the teacher values may affect such synaptic modi ficat ions as specified by the set of weight ing vectors of th e interconnect ions. Translat ed to art ificial neurons, the noise considerat ions refer to inducing an offset in the convergence perform ance of t he network in st riving to reach t he goal or objective value via the supervised learn ing pro cedure implemente d . The dynamic response of a learning network when the target itself changes with t ime can be studied in the information-theoreti c plane and th e relevant nonlin ear (stochastic) dynamics of the learning pro cess can be specified by t he Fokker-Planck equat ion, in terms of a condit ional ent ropy(or mutual informat ion) based error measure elucidated from the prob abilities associated with t he input and teacher (target) values. In t his pap er, the logistic growth (evolut ionary aspects) and certain attractor features of t he learning pro cess are described and discussed in reference to neural manifolds using the mathemati cal found ations of st at ist ical dynamics. Computer simulat ion studies on a test multi layer percept ron are presented , and the asymptot ic behavior of accuracy and speed of learning vis-a-vis the convergence aspects of the test error measure(s) is elucidated. /";:;\ 100 t: (""1 .................1"'..,. C H n ........ ,......, .... D .. l .... 1: ....n ...: ..... n .. T..... ... (la) 350 Neelakanta, Abusalal1, Sudl1akar, De Groff, Aalo, and Park